BRANCHING RULES FOR RAMIFIED PRINCIPAL SERIES 
REPRESENTATIONS OF GL(3) OVER A p-ADIC FIELD 



PETER S. CAMPBELL AND MONICA NEVINS 

Abstract. We decompose the restriction of ramified principal series rep- 
resentations of the p-adic group GL(3, k) to its maximal compact subgroup 
K = GL(3, H). Its decomposition is dependent on the degree of ramification 
of the inducing characters and can be characterized in terms of filtrations of 
the Iwahori subgroup in K. We establish several irreducibility results and 
illustrate the decomposition with some examples. 



1. Introduction 

The complex representations of p-adic algebraic groups are of great interest, 
both in their own right and in what they can reveal through the Langlands 
program in number theory. The representation theory of p-adic groups also 
often mirrors the theory for real Lie groups, and it is especially interesting to 
see how analogous results will develop. 

To this end, one goal is to examine the finer structure of representations by 
considering their restrictions to compact open subgroups. The theory of types 
promises that one can classify representations in the Bernstein decomposition 
by identifying among certain representations of compact open subgroups which 
ones they contain. In contrast, in the theory of real Lie groups, the maximal 
compact subgroups have a crucial role, encoding as they do all the topology of 
the group, and one classifies irreducible unitary representations by classifying 
the irreducible Harish-Chandra modules. Our interest is to explore to what 
extent information about the representations of the p-adic group resides in the 
maximal compact subgroup. 

Representations of compact subgroups of p-adic groups are very tangible at 
a number of levels. Firstly, the representations of sufficiently small (exponen- 
tiable) compact open subgroups can all be constructed using Kirillov theory, as 
shown by Howe [H2]. Secondly, each compact open subgroup is pro-finite and 
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consequently its representation theory is largely determined by the representa- 
tion theory of Lie groups over finite local rings. Finally, any admissible repre- 
sentation of a p-adic group decomposes with finite multiplicity upon restriction 
to a compact open subgroup and so one can expect to recover information 
about the original representation by examining these constituents. 

That said, the maximal compact subgroups are not exponentiable so Howe's 
theory does not apply; and furthermore little information is known in general 
about the representation theory of Lie groups over local rings. Therefore one 
objective of this study is to provide some interchange between the representa- 
tion theories of p-adic groups and of Lie groups over local rings. 

Let k be a p-adic field and denote by 3J its integer ring. In this paper we 
consider the group G = GL(3, k) and let K = GL(3, 3?) be a maximal compact 
subgroup. In |CN| . the authors considered unramified principal series represen- 
tations and showed how their restriction to K decomposed as per the double 
cosets in K of smaller compact open subgroups Cc (defined in Section [2|). In 
|CN| , the added assumption that the inducing character was trivial implied that 
every double coset supported an intertwining operator of the representation, 
an assumption we relax here. 

This paper is organized as follows. In Section [2] we set our notation and 
recall some necessary results from [CN]. The key calculation for determining 
the decomposition is the determination of the double cosets in Cc\K/Ca which 
support intertwining operators for the restricted principal series representation; 
this is the main result in Section [3j We go on to consider questions of irre- 
ducibility in Section H] and conclude with several examples to illustrate these 
decompositions in Section [5l 

The question of parameterizing double cosets of B in K, and more generally 
of the subgroups C(^n,n,n) ill has been visited and solved by several authors 
with various goals in mind. In |QPV| the goal was to look at which Bruhat 
decompositions would be independent of the characteristic of the residue field; 
the answer was that only GL(2, k) has this property. This implies, in particular, 
that the decomposition of principal series is essentially independent of p for 
GL(2,k) (see [HlISi]) but will depend on the properties of the residue field in 
all other cases. 

Several authors have considered related questions on the decomposition of 
representations of p-adic groups upon restriction to a maximal compact sub- 
group. These include the works on Silberger on GL(2, k) [S^, the second author 
on SL(2,k) [Nj, and Bader and Onn [BO] on the Grassmann representation 
of GL(n, k). Gregory Hill has also constructed classes of representations of 
GL(n, 31) in [Hij : a key part of his results was the determination of the double 
cosets of the subgroups C'(ojj) in K. 

2. Notation and Background 

Let k be a p-adic field of characteristic and residual characteristic p. Let q 
denote the number of elements in the residual field of k. We assume throughout 
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that p > 2 and q > 3. Denote the integer ring of k by 3i and the maximal ideal 
of Ji by y. Choose a miiformizer vr and normalize the discrete valuation on k 
so that val(7r) = 1. 

Let G = GL(3,k) and let K = GL(3,3i). Write Tq for the diagonal torus 
in G and Bq for the upper triangular Borel subgroup. Write T = Tq r\ K and 
B = Bq n K for their intersections with K. 

2.1. Principal series and Posets. Let XG be a character, not necessarily 
unitary, of the torus Tq and extend it trivially over the subgroup Bq] then 
the (normalized) induced representation (/>g = Ind^^XG is a principal series 
representation of G. We consider its restriction to K. Writing x = Xg\t and 
(j) = (I)g\ki we have that cj) = Ind^x since K \s good maximal compact. The 
principal series representation is called ramified if x 7^ 1- The unramified case 
was considered in [CNj . 

Given a ramified character XG of Tq-, we may write it as XG = {xiiX2,Xz) 
for characters : k^ — > . Recall that the conductor of a character Xi of k^ 
is the least m > such that 1 + 5"" C ker(xi); thus we make the convention 
that cond(xi) = if and only if Xj la^x = 1. 

The use of normalized induction implies that Ind^^x — Ind^^x'" for any w 
in the Weyl group of G, so we may reorder the characters Xi in a convenient 
way. Moreover, if is a character of k^ and ip ■ x = (V'Xi) V'X2) V^Xs); then 
Ind^gV' ■ X = ° det)Ind^^x- It follows that we may assume that xi = 1 and 
that 

< M = cond(x2) < cond(x3) = N. 
Then cond(xiX2 ^) ~ ^ ^^'^ may furthermore assume that cond(x2X3 ^) = 
cond(xiX3^) = N. Define m = {M,N,N). We wih assume throughout that 
X 7^ 1, so in particular X3 7^ 1 and > 0. 

Let T = {c = (ci,C2,C3) G Z'^ : < ci, C2 < C3 < ci + C2} and note that 
m = (M, N, N) £ T. Then T is a poset with c ^ d if q < for all i. We are 
particularly interested in the subposet Tni = {cGT:c^m}. 

Given c G T, we define a subgroup Cc by 





3? 


01 








yC3 




01 



Then Cc ^ Ca if and only if c ^ d. 

Let Kn denote the nth principal congruence subgroup of K, that is, the 
normal subgroup of K consisting of all those matrices which are equivalent to 
the identity matrix modulo J"". Then for all c G T we have Cc D Kc^. 

Subrepresentations. Let x be the restriction to T of a character of Tq, with 
the above conventions; then in particular xi = 1- If c G 1^, then we can 
extend x to a character of Cc, denoted Xc or simply x if there is no possibility 
of confusion. Namely, given g = (gij) G Cc, we define Xc{g) = X2(522)X3(ff33)- 
One verifies directly that this is multiplicative exactly when ci > M and 02,03 > 
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Definition 2.1. For each c € T^, set Uc = Ind^^Xc- 

We have from [CN] that dimC/c = {q + l){q'^ + q + 1)5^=1+^2+^3-3 ^^^^ > 
and dimC/c = {q'^ + q + l)g^(^i+'=2-i) jf exactly one of ci or C2 is zero. The 
representation Uc is naturally a subrepresentation of in fact, it is contained 
in the subspace of Kc^-&xed vectors of (f>. Consequently, one may also view Uc 
as a representation of the finite group K/Kc^. 

If c ^ d then we have Ua C.Uc, so set 

Vc = Uc/ Yl 

This quotient can be identified with a summand of (j). These summands are 
the building blocks of the decomposition of (j) that we wish to study, so let us 
refine our description of Vc ■ 

Let c E Tm. If C1C2 7^ 0, define for each i € {1,2,3} the triple Cjj} = (ci — 
Sii, C2 - Si2, C3 - 6i3) G Z^; if ci = then only consider C{3} = (0, C2 - 1, C2 - 1); 
and if C2 = then only consider c^^y = (ci — 1, 0, ci — 1). Set Sc = {i : Cjjj £ T^}; 
then for all d ^ c such that d € T^, there is some i £ §c such that d ^ Cjjj. 

Further, let C0 = c and for each non-empty I C 8c define cj = max{d G 
d ^ Cjjj for all i € /}; this set contains m and so is nonempty. For example, 
if m = (0,0,0) and c = (2,3,4) then 0^2} = (1,2,3) since (1,2,4) ^ T„. We 
have the following result from |CNj . 

Theorem 2.2. For any c we have 
(2.1) [K]= 

where [V] denotes the equivalence class of V in the Grothendieck group of K. 

Since the Uc are essentially induced representations of finite groups, the 
dimension !J(C/c5 f^d) of the space of intertwining operators between Uc and U^ 
is equal dim^K(xc5Xd) where 

:K(Xc,Xd) = {/: K - C: f{gkg') = Xc{9)f{k)x^{9')^9 G ^0,5' G ^4. 

As an immediate corollary of the above theorem we therefore have an effective 
means of determining the number of intertwining operators between the various 
quotients Vc- 

Corollary 2.3. Let c,d E Tj,. Then the dimension of the space of intertwining 
operators between Vc and V^ is 

m,v^)= E (-i)i'i+i''i5(f/c,,^d,). 

7C§c,JCSd 

2.2. Distinguished double coset representatives of Cc\K/Cd- We recall 
the parametrization of representatives for the double coset space Cc\K/Ca, as 
given in [CN] . 

Let = {a = (01,02,03) G Z^: 1 < 01,02 < 03}. Given c G T, define 
c G T n by Cj = max{cj, 1} for each i. 
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(2.3) Tc,d = < 



Definition 2.4. For any c,d € T, set 

(2.2) Tc,d = {aST"*^: a^c, a^d and as < min{ai + C2,di + 02}} 

with the fohowing exceptions: 

'{(1,1,1)} if c or d equals (0,0,0); 

{(1, a, a): a < min{c2, ^2}} if 02^2 > and ci = di = 0; and 
{(a, 1, a) : a < min{ci, di}} if cidi > and C2 = d2 = 0. 

Next, for a G Ted set min{a} = minjai, 02, 03}. Then we define 

(2.4) a(c, d) = max{0, min{ai, a2, 03 — ai, 03 — 02, c — a, d — a, 

ai + C2 - 03, di+a2- 03}} 

and 

(2.5) a(c, d)' = max{0, mm{d3 — 03, C3 — 03, ci — oi, ^2 — 02}} > a(c, d). 

For X £ define val(a;) = min{val(y) : y + = x}; then (3i/y'=)^ = 

{x G 3^/^^: val(x) = 0}. We set 

'(r,j/ya(c,d))x if ai+ 02^^03; 

,U=/^'(1 + 7r*3?^) n (3l/T<='<*)+0^ n (3?/y^('=''i)')x if ai+a2 = 03. 

In other words, in this latter case, if val(x — 1) = i > then x and y represent 
the same element of ^ exactly when val(j; — y) > min{f + a(c, d), a(c, d)'}. 

Definition 2.5. Let c, d G T. Enumerate the elements of ~ 5''^ as 

W = {l,Si,S2,SiS2,S2Si,Wo} 

where Sj is the transposition (i i + 1) and wq is the longest element. Define a 
subset Wc,d of W as 

if c,d>i (1,1,1); 
{l,si,wo} if cidi{c2 + 6^2) > and 02^2 = 0; 
{1, S2,wo} if cidi = and (ci + di)c2d2 > 0; 
{1, tfo} if C1C2 = and ^1^2 = but (ci + C2){di + ^2) > 0; 
{1} if c = (0,0,0) or d = (0,0,0). 

The following theorem is proven in [CNj . 

Proposition 2.6. Let c,d G T. A complete set of distinct double coset repre- 
sentatives Rc,d of Cc\K/Cd is 

Fi.c,d = 1^ R^d 

WGWc,d 

where for w G VFc,d w;e define R^^j as follows. 



c,d 



1 










1 





xtt"-'-" 


7r"2 


1 



: a G Tc,d, x G X^^ V ; 
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(ii) r; 



si 
c,d 



(Hi) Rf2 



c,d 



< S 



(v) r; 



S2S1 
c,d 



S2S\ 



(vi) R"'" 



c,d 








1 





— 




1 












1 






1 


o" 


= 


7r/3 


1 




1 









" 





i 


) ^ 




1 














1 












1 





) ^ 




7r° 


1 






1 








"0 


l" 


■ 







1 




1 


1 










1< a <mm{d2,C3} 
1 < (3 < mm{c2, d'i] 
—ci < 13 — a < di 
1 < a < mm{di,C3} 
1 < (3 < min{ci, ds} 
-C2 < (3 - a <d2 

1 < a < min{(ii ,02}^ ; 
1 < a < minjci, 1^2} > ; 



3. Determination of the set of double cosets supporting 
intertwining operators 

To understand the space of intertwining operators of the finite-dimensional 
representations Uc given in Definition 12.11 we construct bases for the spaces 
IK(xc; Xd)- That is, for any c, d G 1^ we must identify among the double cosets 
enumerated in Proposition l2.6l those which support intertwining operators of Uc 
with Ud- Denote the subset of these cosets by Sc,d ^ Rc,d) and write I{Uc, Ua) = 
dim!K(xc5Xd) = |Sc,d|- (Note that in the case that x = 1) which we continue 
to exclude, Rc,d = Sc,d and there is nothing to show.) 

If a G then define a°P = (03 — 02, 03 — 01, (03 — oi) + (03 — 02)). 

Theorem 3.1. Let c,d € T^, with m y (0,0,0) as before. Then a set of 
representatives for double cosets in Cc\K/Ca supporting elements of'K{xciXd) 
is 

Sc,d = 1^ 

where the subsets S^^j C Rjf^^ are defined as follows. 

(i) d is the set of all ta,a; with a € Tc,d o-nd x € X^ ^j such that one of the 
following holds: 

(1) ai > M and 02 > A^; or 

(2) ai < M and 02 > iV and: 

(a) ai + a2 < 03 and M < min{c — a, d — a, C2 + ai — 03, di + 02 — 03}; 
or 

(b) oi + 02 > 03 and M < min{c - a°P, d - a°P}; or 

(c) ai + a2 = 03 and M < min{c — a, d — a} and val(x — 1) < 
a(c,d)' - M; 

or 
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(3) ai > N and a2 < N and the same conditions (a),(h),(c) with M 
replaced by N; 
(ii) S^^^ is the set of all s^^'^^ such that 

N < a < min{d2, C3} - M, 
N < (3 < min{c2, (is} - M, and 
M -ci< 13 -a <di-M; 

(Hi) S*^^ is the set of all s'^'^^ such that 

N< a < min{di,C3} - A^, 
N < P < min{ci,d3} - N, and 
N -C2< P-a <d2-N- 

(iv) and S^j = for all other w G Wc,d o.iT'd for any w ^ W^d- 

Proof. Let us first show that none of the cosets represented by elements of 

R^^^^ U Rc^l^ U f^c'd can support intertwining operators. Choose an element 
b € "Ji^ such that xsi^) 7^ 1- Set g = diag(6, 1, 1) and g' = diag(l, 1,6); these 
are elements of Cc and for any c, d G T^. One verifies that = Sis''2^^ g', 

gwo = wog' and S2S^^^ g = g's2S^^\ but that x{9) ¥^ xi.9')- Consequently none 
of these representatives are in Sc,d- 

From now on, let us adopt the notational convention that if 5 = {gij) G Cc 
then (721 = 72i7r'^i, _ 'j^2Tt'^^ and 531 = 73i7r'^3; gQ g> ^ (j^ would have 
g'21 = 721^*^^ and so forth. Moreover, given a coset representative h G Rc,d and 
a pair of elements g E Cc and g' G Cd such that gh = kg', we will call {g,g') a 
coset pair. 

Suppose now that {g,g') G Cc x Cd are a coset pair for the representative 
-yyg determine directly that the matrix coefficients of g and g' satisfy 

'^^+"-^ + 732vr'=-^-7^ivr'^^-^ 



with the remaining coefficients given by 

511 = 522 - 523^" 5n = 522 + 5237r^ 

512 = 721 tt'^' -523^^ 5i2 = 5237r" + 72i7r'=i 

513 = 523 5'l3 = 523- 

This allows us to compare 

Xcig) = X2(522)X3(533) 

= X2(533 - 5237r^ - 72i7r''^+"-^ + 7327r^^-^ - 73i7r''^-^)X3(533) 





522 


= 533 


- 523 TT^ 


-721^' 


(3.1) 


522 


= 533 


- 523 vr^ 


- 72171"' 




533 


= 533 


- 5237r^ 


- 52371"' 
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with 

Xdig') = X2(522)X3(533) 

XsigSS -52371-^ -523^")- 

It follows that whenever M < min{ci — a+/3, di+a— /?, C2— /?, ^2 — a, C3 — q, ^3 — 

/?} and < mm{a, (3}, then Xcls') = Xdls'); ^^'^ so s^"'^^ G Sc,d. Conversely, 
when these inequalities are not satisfied, and additionally a, /3 > 1, then we can 
use the relations above to construct a coset pair {g,g') on which the characters 
do not agree. This proves part (ii); the proof of part (iii) is analogous and is 
omitted. 

To prove part (i) of the theorem, suppose ta,a- G d ^^'^ l^t {g, 5') G Cc x Cd 
be a coset pair such that gtg^^x = ia.,xg' ■ To simplify notation, set = 7r'^i+"2 _ 
xn""^ . One calculates directly that the matrix coefficients of g and g' satisfy the 
relation 

(3.2) 

+ 732r.vr»i+'^-"^+ 732^vr"i'"'' 

and all other matrix coefficients are determined by the equations 

g'u = 922+523X7r«^-'^^+72ivr^^"'^^-72ivr''^""^ 
511 = g'u - 5i27r''i _ g^^xTr''^ 

522 = 922-512^"^ -513^"'+"' +523^"' 

533 = 922 - 5i2r.7r-«^ - 732vr'=^~"^ + 732^^'""' 
533 = 533 - 513 + 523 vr''^ , 

together with = 5i2 + 5i37r''2, g[^ = gi^ and 533 = 523 - gi?.'^"'^ ■ Note that 
in this case, as opposed to the one for s^"'^^ above, although any solution (with 
coefficients in "X) of ()3.2|) gives a pair of matrices {g,g') satisfying the relation 
gts,,x = ta^xg' 1 it must be additionally verified that g and g' are invertible in K. 
Now, given a coset pair [g, g') we have 

(3.3) Xc{g) = X2(522)X3(533) = X2(522)X3(533 " 513^^ + 5237r''2) 

while 

(3.4) Xd(5') = X2(522 - 5i2vr'^^ - 5i3vr"^+"^ + 523vr"^)X3(533)- 

Hence these characters agree whenever ai > M and a2 > N , proving part 
(i)(l)- 

Now suppose oi and 02 are both less than A^. Choose a pair (512 , 523) G x 3i 
of minimum valuation satisfying (7237r'^^ = 5i27r'^^, set 513 = 721 = 721 = 731 = 
731 = 732 = 732 — ^-^d set 522 = 1- These are easily seen to define a coset 
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pair {g,g') G Cc x Ca- Since val((7i2vr''i) = val((723vr"2) = max{ai,a2} < N = 
cond(x3), we have Xc{g) / Xd{g') and it follows that ta,a; ^ S^^jj. 

There are exactly two cases left to consider: when oi < M and 02 > A'', 
or when ai > N and 02 < A^. Comparing (j3.3p and (13. 4p . and noting that 
max{ai,a2} < minjas, va^r^^)}, we deduce that 

(A) if ai < M and 02 > A^, then ta,a; G Sj^^j if and only if val(gi2vr''i) > M for 
all coset pairs {g,g')', and 

(B) if ai > A^ and a2 < N, then ta,x e ,j if and only if val(g23vr"2) > A^ for 
all coset pairs {g,g'). 

Consider case (A), that is, assume that oi < M and 02 > A^. 

If val(gi2'/r'*^) > a2 > N then we are done; otherwise, the term with least 
valuation on the left hand side of (j3.2p is gi2'^'^^xrx. Comparing with the right 
hand side, we deduce val(5i2vr"i) + val(r^) > a where 

a = min{ci + a2,di + 02 - 03 + val(r2:),ai + C2 - as + val(r2;), ai + d2, 
C3 — 03 + «i + 02, d-i — as + ai + 02}. 

It follows that if a > M + val(r^), then ta,a; £ Sc,d by (A) above. Restating this 
condition in the three cases ai + 02 < as, ai + 02 > as and ai + a2 = as yields 
the conditions described in part (i)(2)(a,b,c) of the theorem. 

Conversely, suppose a < M+val(r^) and set gis = 923 = 0. Choose a term of 
least possible valuation on the right hand side of (j3.2p : set its coefficient (either 
7jj or j'-p for some i > j) to be vr'^i^" if a < ai + val(r^) and 1 otherwise. 
Then set the remaining coefficients of the right hand side of (|3.2p equal to zero 
and solve for 1712, which is now necessarily in Jl^ . Take 322 = 1 and solve for 
the remaining coefficients. This results in a coset pair (g, g') € Cc x Cd such 
that val(gi2vr'*i) < M, so by (A) we conclude ia,x ^ Sc,d) as required. 

A similar argument establishes condition (i)(3) of the Theorem, following 
case (B), above. □ 

Let us conclude this section by deriving some consequences of Theorem 13. li 
The first, which is immediate, is a convenient restatement of the theorem in a 
special case. 

Corollary 3.2. Set c = (n,n,n) for n > N. The space of intertwining opera- 
tors of Uc = with itself has basis parametrized by 

where 

(i) is the set of all ta,x such that I < 01,02 < as < n, x ^ ^ and one of 
conditions (1), (2) or (3) is met: 

(1) a ^ m; or 

(2) ai < M and 02 > N and: 

(a) ai + 02 < as < n — M, or 

(b) ai + a2 > as and ai + 02 > M — n + 2as, or 
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(c) ai + a2 = aj, < n — M and \al{rx) < n — M; or 
(3) ai> N , a2 < N and the same conditions (a),(h),(c), with M replaced 
by N, are satisfied; 
(ii) S^i = {s^"'^^ : N <a,/3 <n- M}; 
(Hi) = {s^"'^^ : N <a,(3 <n- N}; 

(iv) S^^l"2 = QS2S1 ^ QWo ^ 0_ 

Our second corollary will be relevant for the purposes of calculating J(Vc, T4) 
in Section m 

Corollary 3.3. Suppose that c,d, c',d' € with c < c' and d ■< d'. Then 

Proof. By identifying X^^ with a set of coset representatives from "R^ in a 
suitable manner, one easily sees that Rc,d ^ ^c',a'- Furthermore, it is clear that 
the list of constraints on elements of S in Theorem 13.11 can only become less 
constrictive as c or d increase. □ 

In particular, it makes sense to ask, for a given distinguished double coset 
representative g € Uc,dRc,d) whether there exist c, d € for which g € Sc,d. 

Theorem 3.4. The double cosets which support self-intertwining operators of 
Uc, for some c € Tm, are represented by 

s = u Sc,d = U 

where 

(i) = {ta,^: 03 > max{ai,a2} > iV, x G 3^^}; 
(ii) S^i = {4°'^^: a,/3 > N]; 
(Hi) = : > AT}; and 

(iv) S^i'*^ = s^asi = ^ 0_ 

Moreover, up to identifying ta,x ^a,?/ whenever x and y have the same image 
in (J for c, d sufficiently large, these cosets are all distinct. 

Proof. This follows from Corollary 13.21 by allowing n to grow without bound. 
Note that when c = d = {n,n,n), we have simply a(c,d) = mm{ai, 02,03 — 
ai, 03 — a2,n — 03} and a(c, d)' = n — 03. □ 

4. Irreducibility 

The results of the preceding section allow us to restate Corollarv l2.3l in terms 
of the sets Sc.d- That is, for any c, d € Tm, we have 



IiV,,Va)= V (-Ijl^l+l-'l |S, 



/ ^ \ i-->C7,dj| 



The irreducibility of {7^ = 14 is known from Howe's work [Hl^ Theorem 
1]. In this section, we demonstrate that this extends to many, but not all, of 
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the quotients which are "extremal" in the sense that they have few immediate 
descendants in the poset Tm- 

We retain the notation of the previous sections. 

Theorem 4.1. For each n € Z with N < n < N + M , the K-module V(Af 7v,n) 
is irreducible. 

Proof. Set c = (M,N,n). n = N then Sc = 0; otherwise, §c is a singleton 
corresponding to the triple {M,N,n — 1). By Corollary 12.31 and induction, it 
thus suffices to show that |Sc,c| = n — N + 1. 

If M = then Wc,c = {l,wo} so S^i,, = S^f^ = 0. If M > then min{c2, csj- 
iW = iV - M < iV and min{ci, C3} -N = M-N < N so again S^]^ = S^^^ = 0, 
regardless of the value of n. Thus Sc,c = c- 

Now let ta,x £ c! so one of Theorem 13. iT ilfl). (2) or (3) applies. If it were 
(2), then ai < M and a2 > N imply that M > and a2 = N so neither case (a) 
nor case (c) could apply since M > C2 — 02 = 0. Were case (b) to apply, then 
M < min{c — a°P} would imply that a2 = = N and so N — (03 — ai) = oi 
which is not greater than or equal to M, a contradiction. We similarly deduce 
that case (3) cannot apply. This leaves case (1), which consists of the elements 
ia,x with a = {M,N,m), N < m < n and a; € X^^, each of which support an 
intertwining operator. For each such a, we have a(c, c) = a(c, c)' = 0, so in 
fact |X^c| = 1. The desired conclusion follows. □ 

Theorem 4.2. V(^fn^n,n+m) ^-^ irreducible for each m > M and n > N. 

Proof. We first consider the case that c = (m, n,m + n) with m > 1. Then §c 
is a singleton corresponding to d = C|3| = (m,n,m + n — 1). Hence J(C/c) K:) = 
J([/c, Uc) — "JiUc, Ud) and it suffices to show that |Sc,c \ Sc,d| = 1. 

First note that since ci = di and C2 = ^2, and that both these are at most 
ds < C3, we have S^^c = S^.d for each w e W \ {1}. 

Next, note that Tc,c \ Ted consists of the single element {m,n,n + m) = c. 
Since j,| = 1, there is a unique distinguished double coset of the form tc,x G 
it is clearly in Sc,c- We claim that this is the only element of rJ <, \ a- 
Namely, let a G Tc,d- Since < ^3 — 03 = (di + (i2 — 1) — 03 = {di + 02 — 03) + 
{d2 — 02) — 1, it must be true that d^ — > min{di + 02 — 03,^2 — 02} so 
necessarily a(c, c) = a(c,d). If furthermore 01+02 = 03, then a(c, c)' = a(c,d)' 
by the same reasoning. Hence X^^, = X^^ for all such a, as claimed. 

We next claim that ^ n rJ ^ = Namely, given tg^^x ^ S^ c Pi ^, since 
ds — as > minjdi + 02 — 03, ^2 — 02} = min{ci + 02 — 03, 02 — 02}, we see that 
all of the conditions set out in Theorem I3.1l fil are unchanged in passing from 
the pair (c, c) to the pair (c,d). Hence ta,a: £ S^d- 

This shows, for the case m > 1, that Sc,c \ Sc,d is a singleton from which we 
deduce the irreducibility of Vc. 

When c = (0,n,n), we have instead §c = {3} corresponding to d = C{3} = 
(0, n — 1, n — 1). We have Sc,d = ^, Sc,c = <. and neither of the cases (i)(2) 
or (i)(3) of Theorem 13.11 can apply. It thus follows easily that |Sc,c \ Sc,d| = 1 
in this case as well. □ 
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Theorem 4.3. V(m,ra,max{n,m}) ^-5 irreducible for each m > M and n > N . 

Proof. Suppose first that m > max{M, 1} and n > N, and that max{m, n} = 
n. Then c = {m,n,n) and Sc = {1,2} with the corresponding triples C{ij = 
(m — 1, n, n), C|2} = {m, n — l,n) and C{i 2} = (m — 1, n — 1, n). We compute 
the alternating sum 

(4.1) I{Uc,Vc) =I{Uc,Uc) -I{Uc,Uc^,^) -I{Uc,Uc^,^)+I{Uc,Uc^,^^^) 
as a sum of differences by defining 

•^1 = ^C,C{i} \ Sc,C{i^2} • 

Thus we have I{Uc,Vc) = \Ao\ — \Ai\. We use (d, d') to denote either of the 
pairs (c, C|2}) or {c^iy, C{i_2})) for ease of notation. 

Suppose first that s^"'^^ S Sc,d \ Sc,d'- Then, comparing the constraints on a 
and P in Theorem IS.lT ii) for d and d', we see that necessarily a = ^2 — M = 
n — M and 

max{iV, M -ci + a} <(3 < min{di - M + a, C2 - M}. 

Since di > M by hypothesis, C2 — M = n — M < a + di — M so these inequalities 
simplify to max{A^, n — m} < f3 < n — M. This constraint on the pair (a, /?) 

is independent of the value of di G {m — l,m} so s^"'^^ ^ -^0 if ^-nd only if 

^ Hence these cosets contribute nothing to the overall sum (14. Ih . 

Now suppose that 52"'^'' € Sc,d- Then Theorem 13. l( iii) implies that (3 — a < 
{ci—N)—N; but this bound is at most d'2—N since ci—N = n—N < n—1 = d^- 
Similarly, di < ci and (5 > together imply that f5 — a > N — ci, regardless of 
the value of di £ {m — l,m}. All other conditions on {a, f3) being unchanged 
in passing from (c,d) to (c,d'), we deduce that 52"'^^ ^ ^c,d'- Hence none of 
these cosets appear in either ^0 or Ai. 

Finally, consider distinguished coset representatives of the form ta,a: £ d- 
First note that 

Tc,d\Tc,d' = {(ai,n,n): 1 < ai < dj, 
and |X^^| = [X^ J, I = 1, since 03 — 02 = 0. Considering which of these are in 
Sc,d, we deduce that these triples give rise to m — M + 1 coset representatives 
in ^0 and m — M of them in Ai- 

Suppose now that a S Tc^d'- Since < ^2 — a2 = ds — 1 — 02 = (ds — 
03) + (03 — 02) — 1, we have ^2 — ^2 ^ min{(i3 — 03, 03 — 02} and so it follows 
that a(c,d) = a(c,d'). Similarly, if ai + 02 = 03 then 02 < 03 implies that 
^2 — 0.2 > ds — as so a(c, d)' = a(c, d') . Hence for all a G Tc,d' we have 

■'^c,d — ■'^cd'- 

So suppose ta,a; ^ ^ H ^j/. We first note that if fa,a- falls under any of the 
conditions (2a), (2c), (3a) or (3c) of Theorem 13 -H then the inequality 02 < 03 
implies ^2 — 02 > d^ — a^. Consequently, this condition is unchanged in passing 
from d to d' and so ta.x £ Sc,d'- Similarly, if ta,x falls under condition (3b), 
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then 02 < as so d'2 — (03 — ai) > ds — (03 — ai) — (03 — 02); again we deduce 
t&.x G Sc,d'- So none of these occur in either ^0 or ^1. 

On the other hand, if fa.x falls under condition (2b) for the pair (c, d), then it 
fails (2b) for the pair (c, d') exactly when 03 = 02 > N , di > M, (i2 — (03— «i) = 
M and 1 < ai < M. Hence, noting also that this condition is independent of 
the choice of di € {m — l,m}, all such ta,a; he in both Aq and Ai- 

We deduce that |^o| — = 1 so the quotient Vc is indeed irreducible. 

The case for m > n follows by an analogous argument, where we interchange 
the roles of C|i} and C|2} throughout. 

It only remains to show the case where m = 1 and M = 0. In this case, 
c = (1,71, n) with n > > 1 so we have c^iy = (0, n,n), C|2} = (l,'^ — 
l,n) and C|i 2} = (0,n — l,n — 1). Define ^0 and ^1 as above. Since ci — 
ai = for all a G Tc,d, for any d, and since cases (i)(2) and (i)(3) cannot 
occur, the analysis is much simplified from the above. We readily see that 

Aq = {si"'"~^\s^'''"\t(i_„^„)^i} whereas Ai = {t(i,n,n),i' *(i,n-i,n),i}- Thus we 
conclude again in this case that Vc is irreducible. □ 

Recalling that V^'^ ~ U(^n.n,n) we immediately have the following Corollary. 

Corollary 4.4. If n > N then the quotient V(„ „ „) is the unique irreducible of 
maximal dimension in V^" . 

The strict inequalities in Theorem 14.31 are necessary, as the following propo- 
sition shows. 

Proposition 4.5. 

(1) Let c = (M, n, n) with n> N . Then 



(2) Let c = {n,N,n) with n>N. Then 



n- N + 1 ifn<M + N; 
M + 1 ifn>M + N. 



n- N +1 ifn<2N; 
N+1 ifn>2N. 



Proof. To prove part (1), let c = (M,n,n) with M > and n > N. Then 
§c = {2} corresponding to the triple C{2} = {M,n — l,n). We first compute 
J(f7c,Vc) = |Sc,c \Sc,c{2}l- Tor ease of notation, set (d, d') = (c,C|2}). 

It is easy to see that sS" ^'^ G Sc,d \ Sc,d' if n - M > A^, whereas 

^c,d — ^c,d' ~ ^• 

Of the elements in Tc,d \ Tc,d' = {(ai,n,n): 1 < ai < M}, only (M, n, n) 
gives rise to a representative in Sc,dj and then exactly one, which we'll denote 

t(M,n,n),l- 

For each a G Ted', we have 02 < n and 03 < n. Since a(c,d) < minjas — 
a2,n — 03,(^2 — 02} and 03 < ^2 we deduce that if a(c,d) 7^ a(c,d') then 
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necessarily 03 = n and 03 = 02, a contradiction. Similarly, a(c,d)' does not 
depend on the value of d2 ■ Hence = ^j, . 

So suppose ta,a' £ (sj (J n Rc,d') \ Sc,d'- It does not fall under case (i)(l) of 
Theorem 13.11 since this case is independent of d'; nor can case (i)(3) occur since 
fli < M. In cases (i)(2)(a) and (c), the right hand side can depend on the value 
of d2 £ {n — 1, n} if and only if 02 = 03, contradicting the hypotheses. In case 
(i)(2)(b), which holds only if 02 > N, we must have that 03 — 02 = or else 
the right hand side is less than M. It follows that the right hand side depends 
on the value of ^2 exactly when a2 = > N and n — (03 — ai) = M; in each 
of these cases |X^(j| = |X^jj/| = 1 and ta,i G ^ca \ Sc,d'- 

We conclude that when M > 

Sc,c \ Sc,c{2) = {t(M-k,n-k,n-k),i : < k < mm{M - 1, n - A^}} 
U {s^^-M,n-M) . if „ _ ^ > 

A simpler analysis, which we consequently omit, allows us to further deduce 
that Sc|2},c = Sc{2},c{2} s° "^{UcVc) = J(Vc,Vc), and this has the value 
stated in the theorem. 

When M = 0, we have instead c = (0,n,n) and C|3} = (0,n — l,n — 1), 
and Sc,d = Sj^. Since neither (i)(2) nor (i)(3) of Theorem 13.11 can apply, and 
a(d, d') = a(d, d')' = for all choices of d, d' € {c,C|3}.} and for all a G Td,d'i 
we readily conclude that Sc,c \ Sc,C{3} = {*(i,n,,n),i}- Hence the quotient Vc is 
irreducible in this case. 

To prove part (2), let c = {n,N,n) with n > N. Then 8c = {1} with 
corresponding triple (n — l,N^n). Reasoning as above, we deduce readily that 
^(n-Ar,n_7V) ^ ^^^^ ^c,c<^i} whenever n>2N and that S^]^ = K]c{^} = ^■ 

Set (d, d') = (c,C|i}.). Note that Tc,d \ Tc^d' = {{n.,a2^n): 1 < a2 < N} and 
each of these has |X^jj| = 1. These triples thus give rise to only one coset in 
sj ,j, namely that represented by t(^n,N,n),i- 

Of those a € Tc^d'; one sees as above that X^ ^j = X^^j;. For such a triple a, if 
ta.,x G Sc,d\Sc,d' then it falls under case (i)(3)(b) of Theorem 13. II and we deduce 
as above that N < ai = a-^ <n and n — = N — a2- These conditions further 
imply that \^cd\ — ^ meaning each such triple gives rise to a unique double 
coset. 

We conclude that 

Sc,c \ Sc,c{i} = {t{n-k,N-k,n-k),i : < /c < min{n - N,N - 1}} 
U {4"-^'"-^) :iin-N>N}. 

It is readily verified that Sc^ij,c = Sc{^j,c{i}j and so 'J{Uc,Vc) = J(Vc,Vc). 
Counting the double cosets in the expression above yields part (2) of the theo- 
rem. □ 
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5. Examples 

We conclude the paper with two examples to illustrate the results in Sec- 
tion H 



(4,4,4) 
1 



(4,2,4) 

3 



(3,2,4) 
1 



(3,2,3) 

2 



Figure 5.1. Reducibility of K for Vy. with M = 2 and iV = 2. 



Quotient 


Dimension 


%,4,4) 


q'{q - 


lya 


^(3,4,4)) ^(4,3,4) 


qHq- 


Ifa 


^(3,3,4) 


(f{q - 


Ifa 


^(2,4,4), ^(4,2,4) 


q^{q- 


l)a 


^(2,3,4), ^(3,3,3); ^(3,2,4) 


q\q- 


Ifa 


^(2,3,3) J ^(2,2,4) , ^(3,2,3) 


q\q- 


l)a 


^(2,2,3) 


q^{q- 


l)a 


^(2,2,2) 


q^a 





(3,4,4) (4,3,4) 
1 1 



(2,4,4) 

3 



(2,3,4) 



(2,3,3) 

2 




(3,3,4) 



(3,3,3) 

1 



(2,2,4) 



(2,2,3) 
1 



(2,2,2) 

1 




Table 5.1. Dimensions of Vc for with M = 2 and N = 2. 
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(1,4,4) (2,3,4) (3,3,3) (3,2,4) 
2t g-2 t 1 1 



(1,3,4) (2,3,3) (2,2,4) (3,2,3) 

1 * 1 1 * 2 




Figure 5.2. Reducibility of and equivalences between Vc for 



with M = 1 and iV = 2. 



Example 5.1. Suppose that M = N = 2 and let us consider the decomposition 
of V^'^ under K. The values of Z(Vc, V^) are calculated using Corollarv 12.31 and 
Theorem 13.11 with several values identified by Theorems 14.11 14.21 and 14.31 and 
Proposition l4.51 The remaining computations were implemented in GAP [GAPj 
and the results are represented schematically in Figure 15. H as follows. 

Each triple c in Figure [5T] corresponds to the induced representation 11^, 
and the number beneath it is the value of liVcVc). The arrows imply the 
partial order ^ on T; hence the set of all components of the diagram below and 
including c may be identified with the whole of Uc- For reference, we list in 
Table 15.11 the dimensions of the quotients occuring in Figure 15.11 These are 
calculated using Theorem 12.21 We abbreviate a = (g + l)(g^ + g + 1). 

Figure 15.11 reveals several typical features of the iiT-representations Vc ■ For 
example, we note that while many Vc are irreducible, several are not. Besides 
those identified by Proposition 14.51 for whom the number of intertwining op- 
erators grows at most linearly with C3, there exist components such as V(3 3 4), 
for which the number of intertwining operators is a polynomial function of g. 
Such components occur more frequently in V^" as n increases, since they come 
into existence only when \^c c\ is a polynomial in g, that is, when a(c, c) > 0. 

Example 5.2. Consider a character x for which M = 1 and N = 2. Figure [5^2] 
describes a portion of the restriction to -fC of V^, namely, all subrepresentations 
Uc for which Vc has dimension of order or less. In terms of triples, this implies 
that we consider the elements c € 1^ for which ci + C2 + C3 < 9. Again, the 
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C^uotient 


Dimension 


1^(1,4,4) 


q [fl — ija 


^(3,3,3)) ^(3,2,4) 


q\q - \fa 


^(2,3,4) 


q\q-\)[q-2)a 


^(1,3,4)) ^(2,2,4)) ^(3,2,3) 


q^{q - l)a 


^(2,3,3) 


q^{q — 


^(1,3,3) 


q\q-l)a 


^(2,2,3) 


q^q-lfa 


^(1,2,3) 5 ^(2,2,2) 


q^{q-l)a 


^(1,2,2) 


q^a 



Table 5.2. Dimensions of Vc for K. with M = 1 and N = 2. 



number of intertwining operators between each pair of quotients is determined 
by CoroharyE^l 

This example illustrates a phenomenon not present in Example 15.11 For in- 
stance, there are two pairs of isomorphic irreducible representations: V(i 3 4) ~ 
V(2,2,4) (indicated by * in Figure [5^2]) and one of the two inequivalent irreducible 
summands of V(i^4 4) is isomorphic to exactly one of the irreducible summands 
of V(2,3,4) (indicated by f in Figure [52]) • Such pairs of isomorphic irreducibles, 
for distinct triples c,d € T^, can occur only when C3 = ^3, since otherwise the 
corresponding groups Cc and lie in different levels of the filtration of K by 
the normal subgroups Kn, which in turn would imply that Vc and Va cannot 
intertwine as representations of K. 

The dimensions of the representations in Figure 15.21 are given in Table 15. 2[ 
We have again abbreviated a = {q + \){q^ + q + 1) . We conjecture that V(2,3,4) 
in fact decomposes as a sum of g — 2 distinct irreducibles, each of dimension 
equal to that of V(i,3,4), ^(2,2,4) and V(3 2,3). This would be consistent with the 
remaining irreducible in V(-i 4 4) having dimension equal to that of V(-3 3 3) and 

^(3,2,4)- 
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